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Abstract — We consider a real-time streaming system where 
messages are created sequentially at the source, and are encoded 
for transmission to the receiver over a packet erasure link. Each 
message must subsequently be decoded at the receiver within 
a given delay from its creation time. The goal is to construct 
an erasure correction code that achieves the maximum message 
size when all messages must be decoded by their respective 
deadlines under a specified set of erasure patterns (erasure 
model). We present an explicit intrasession code construction 
that is asymptotically optimal under erasure models containing 
a limited number of erasures per coding window, per sliding 
window, and containing erasure bursts of a limited length. 
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I. Introduction 

We consider a real-time streaming system where messages 
are created sequentially at the source, and are encoded for 
transmission to the receiver over a packet erasure link. Each 
message must subsequently be decoded at the receiver within 
a given delay from its creation time. We seek to construct 
an erasure correction code that withstands a specified set of 
erasure patterns (erasure model), allowing all messages to be 
decoded by their respective deadlines. 

In particular, we consider three erasure models: the first 
model limits the number of erasures in each coding window, 
the second model limits the number of erasures in each sliding 
window, while the third model limits the length of erasure 
bursts. For each erasure model, the objective is to find an 
optimal code that achieves the maximum message size, among 
all codes that allow all messages to be decoded by their 
respective deadlines under all admissible erasure patterns. 

We present an explicit intrasession code construction which 
specifies an allocation of link bandwidth or data packet space 
among the different messages; coding occurs within each mes- 
sage but not across messages. Intrasession coding is attractive 
due to its relative simplicity, but it is not known in general 
when intrasession coding is sufficient or when intersession 
coding is necessary. We show that for an asymptotic number of 
messages, our code construction achieves the optimal message 
size among all codes (intrasession or intersession) for the 
first and second models with any given maximum number of 
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Fig. 1. Real-time streaming system for (c, d) = (3, 8). Each of the messages 
{1, . . . , 5} is assigned a unique color. Messages are created at the source at 
regular intervals of c time steps, and must be decoded at the receiver within d 
time steps from their respective creation times. At each time step t, the source 
is allowed to transmit a single data packet of noiTnalized unit size over the 
link. 



erasures per window, and for the third model when the given 
maximum erasure burst length is sufficiently short or long. 

In related work, Martinian et al. ^ provided construc- 
tions of streaming codes that minimize the delay required 
to correct erasure bursts of a given length. Streaming codes 
in which the decoding error probability decays exponentially 
with delay, called tree codes or anytime codes, are considered 
in |4]-[6|. Tekin et al. iTJ considered erasure correction coding 
for a non-real-time streaming system where all messages are 
initially present at the encoder 

We begin with a formal definition of the problem in 
Section [III and proceed to describe the construction of our 
intrasession code in Section |lll] We then demonstrate the opti- 
mality of this code under each erasure model in the subsequent 
sections. Proofs of theorems are deferred to Appendix Ib] 

II. Problem Definition 

Consider a discrete-time data streaming system comprising 
a source and a receiver, with a directed unit-bandwidth packet 
erasure link from the source to the receiver Independent 
messages of uniform size s > are created at the source at 
regular intervals of c G Z+ time steps, and must be decoded 
at the receiver within d G Z+ time steps from their respective 
creation times. At each time step t G Z+, the source is allowed 
to transmit a single data packet of normalized unit size over 
the link. Fig. [T] depicts this real-time streaming system for an 
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instance of (c, d). 

More precisely, each message k G Z+ is created at time step 
(fc — l)c + 1, and is to be decoded by time step (fc — l)c + d. 
The coded data transmitted at each time step t e Z+ must be 
a function of messages created at time step t or earlier Let 
coding window Wk be the interval of d time steps between 
the creation time and the decoding deadline of message k, i.e., 

W^fc^{(fc-l)c+l,...,(fc-l)c + 4. 

We shall assume that d > c so as to avoid the degenerate case 
of nonoverlapping coding windows for which it is sufficient 
to code individual messages separately. 

Consider the first n messages {1, . . . , n}, and the union of 
their (overlapping) coding windows r„ given by 

T^^WiU---UWn^{l,...,{n~l)c + d}. 

An erasure pattern -E C T„ specifies a set of erased data 
transmissions over the link. More precisely, if t G -E, then none 
of the data transmitted at time step t is received by the receiver 
(i.e., the entire data packet is erased); if t G Tn\E, then all of 
the data transmitted at time step t is received by the receiver 
at time step t (i.e., the entire data packet is received without 
delay). An erasure model specifies a set of erasure patterns 
that an erasure correction code should withstand. 

For a given pair of positive integers a and b, we define 
the offset quotient qa^b and remainder ra^b to be the unique 
integers satisfying the following three conditions: 



a = qa,b b + 



a.bi 



qa,b e rafi e {i,...,6}, 



where Zq denotes the set of nonnegative integers, i.e., 
Z+ U {0}. Note that our definition departs from the usual 
definition of quotient and remainder in that rafi can be equal 
to b but not 0. 

III. Code Construction 

We present an intrasession code which codes only within 
each message and not across different messages. We begin 
by specifying the amount of link bandwidth or data packet 
space allocated for the encoding of each message at each time 
step. An appropriate code (e.g., random linear coding, MDS 
code) is then applied to the allocation so that each message 
can be decoded whenever the total amount of received data 
that encodes that message is at least the message size s. 

The allocation of link bandwidth follows a simple rule: the 
link bandwidth at each time step is divided evenly among all 
active messages. We say that message k is active at time step t 
if and only if t falls within its coding window, i.e., t G Wk- 
Fig. |2] shows how much link bandwidth at each time step is 
allocated to each message, for two instances of (c, d). 

For a given choice of {c,d), the messages that are encoded 
at a given time step t G Z+ can be stated explicitly as follows: 
First, we define At to be the set of active messages at time 
step t, i.e.. 



At^{keZ+ -.te Wk} 

= {fc G Z+ : (fc - l)c + 1 < t < (fc 



l)c + d} 



Wi 4 



W2 ^ 



W4 ^ 



5 6 6 



4 5 5 



111222333444555666 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 
(a) (c,d) = (3,9) 



^ t 



W^ 4 



W4 * 



We 



1 2 3 4 5 6 7 



9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 
(b) (c,d) = (3,8) 



Fig. 2. Allocation of link bandwidth at each time step t, in the encoding 
of messages {1, . . . , 6}, for (a) (c, d) = (3, 9) and (b) (c, d) = (3, 8). Each 
message is assigned a unique color. In (a), because d is a multiple of c, we 
have g^j c + 1 = 3 active messages at each time step. In (b), because d is not 
a multiple of c, we have either q^j ^ = 2 or c + 1 = 3 active messages at 
each time step. 



, rr,^ t-d , t-1 

keZ+ : h 1 < fc < h 1 



Treating nonpositive messages 0, —1, —2, ... as dummy mes- 
sages, we can write 

't-d 



At = 



1 



t - 1 



1 



Expressing this in terms of qd,c, rd^c, Qt,c, ft^c yields 

'rt,c - Td. 



At = < qt,c + 1 - qd,c 



■ , 9t,c + 1 



It follows that the number of active messages \At \ varies over 
time depending on the value of rt,c, specifically, two cases are 
possible: 

Case 1: If rt,c < ^d c, then 



1 — c rt , 
-1 < < ^ 



Td., 



which implies that 



c 



= 0, and 



<o, 



At = {qt.c + 1 - qd,c, • • ■ , qtx. + 1} • 
The qd,c + 1 messages of At are therefore encoded at time 



step t, with each message allocated 
bandwidth. 

Case 2: If c > then 



gd,e+i 



amount of link 



Tt r — rd r C — 1 
< — ^ < < 1, 



3 



which implies that 

At = {qt,c + 1 



1, and 



('?<i,c-l),---,gt,c + l}. 

The c messages of Ai are therefore encoded at time step t, 
with each message allocated amount of link bandwidth. 

Note that when d is a multiple of c, we have ^ < ^d.c = c 
for any t, which implies that qd.c + 1 messages are encoded 
at every time step. 

In our subsequent performance analysis of this code, we 
make repeated use of two key code properties; these are 
presented as technical lemmas in Appendix |A] 



IV. Performance under z Erasures 
PER Coding Window 

For the first erasure model, we look at erasure patterns 
that have a limited number of erasures per coding window. 
Consider the first n messages {1, . . . ,n}, and the union of 
their (overlapping) coding windows T„. Let £™ be the set of 
erasure patterns that have z or fewer erasures in each coding 
window Wk, i.e.. 



ccw A 



{ECT„: \EnWk\ < zVfcG {!,...,«}}. 



The objective is to construct a code that allows all n mes- 
sages {1, . . . ,n} to be decoded by their respective deadlines 
under any erasure pattern E £ f ™. Let be the maximum 
message size that can be achieved by such a code, for a given 
choice of {n, c, d, z). 

We observe that over a finite time horizon (i.e., when the 
number of messages n is finite), intrasession coding can be 
suboptimal. The following example shows that an intersession 
code can achieve a message size that is strictly larger than the 
message size achieved by an optimal intrasession code: 

Example (Finite time horizon). Suppose that (n, c, d, z) = 
(3, 1, 3, 1). The maximum message size that can be achieved 
by an intrasession code is |; one such optimal intrasession 
code, which can be found by solving a linear program, is 
as follows (the amount of link bandwidth allocated to each 
message is indicated in parentheses): 
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The following intersession code achieves a strictly larger 
message size of 1 {Mk denotes message k): 
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Using a simple cut-set bound argument, we can show that this 
is also the maximum achievable message size, i.e., s™ — 1. 

However, it turns out that the intrasession code constructed 
in Section |lll] is asymptotically optimal; the gap between the 
maximum achievable message size sj^" and the message size 
achieved by the code vanishes as the number of messages n 
goes to infinity: 

Theorem 1. The code constructed in Section |^ is asymp- 
totically optimal in the following sense: the code achieves a 
message size of 

d—z 

which is equal to the asymptotic maximum achievable message 



size lim„_>oo s™, where y 



(yi,... 

d entries 



, Ud) is defined as 



A 

y = 



1 



(ld,c 



1 



1 



(9d,c + l)'"d,c entries 



qd.aic-Td.a) entries 



The achievability claim of this theorem is a consequence of 
Lemma [T] to prove the converse claim, we consider a cut-set 
bound corresponding to a specific worst-case erasure pattern 
in which exactly z erasures occur in every coding window. 
This erasure pattern is chosen with the help of Lemma |2l 
specifically, the erased time steps are chosen to coincide with 
the larger blocks allocated to each message in the constructed 
code. 

V. Performance under z Erasures 
PER Sliding Window of d Time Steps 

For the second erasure model, we look at erasure patterns 
that have a limited number of erasures per sliding window of 
d time steps. Consider the first n messages {!,..., n}, and the 
union of their (overlapping) coding windows T„. Let sliding 
window Lt denote the interval of d time steps beginning at 
time step t, i.e., 

LtA {t,...,t + d-l}. 

Let £^ be the set of erasure patterns that have z or fewer 
erasures in each sliding window Lt, i.e., 

^sw A |^CT„ : iSnLtl <zV< £{!,..., (n-l)c+l}}. 

The objective is to construct a code that allows all n messages 
{1, . . . , n} to be decoded by their respective deadlines under 
any erasure pattern E £ Ef^. Let s^™ be the maximum message 
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size that can be achieved by such a code, for a given choice 

of (n, c, d, z). 

We note that since f^™ C £™, we therefore have s"^ > s^,". 
For the special case of c — 1, each sliding window is also 
a coding window, and so this sliding window erasure model 
reduces to the coding window erasure model of Section |IV] 
i.e., £^ = £™. Over a finite time horizon, intrasession coding 
can also be suboptimal for this erasure model; the illustrating 
example from Section HV] applies here as well. 

Surprisingly, the constructed intrasession code also turns out 
to be asymptotically optimal over all codes; the omission of 
erasure patterns in £^ compared to f ™ has not led to an 
increase in the maximum achievable message size (cf. Theo- 
rem [T]l: 

Theorem 2. The code constructed in Section \ni\ is asymp- 
totically optimal in the following sense: the code achieves a 
message size of 

d—z 

which is equal to the asymptotic maximum achievable message 
size lim„ 



Proving the converse claim of this theorem requires a dif- 
ferent approach from that of Theorem [T] When d is a multiple 
of c, we need only consider a cut-set bound corresponding 
to an obvious worst-case erasure pattern in which exactly z 
erasures occur in every sliding window, specifically, a periodic 
erasure pattern with alternating intervals of z erased time steps 
and d — z unerased time steps. When d is not a multiple 
of c, no single admissible erasure pattern provides a cut-set 
bound that matches the constructed code; instead, we need 
to combine different erasure patterns for different messages. 
To pick these erasure patterns, we first choose a specific base 
erasure pattern E' (which may not be admissible in general) 
with the help of Lemma |2] We then derive admissible erasure 
patterns from E' by taking its intersection with each coding 
window, i.e., [E' n Wk) E £f^- These derived erasure patterns 
are used in the inductive computation of an upper bound for 
the conditional entropy 



r(n-l)( 



H{X[Wn\E'] M^,X[ 



where Xt is a random variable representing the coded data 
transmitted at time step t, Mk is a random variable repre- 
senting message k, and X[A] = {Xt)t£A- Intuitively, this 
conditional entropy term expresses how much space is left in 
the unerased data packets of the coding window for message n, 
after encoding the first n messages, and conditioned on the 
previous time steps. The nonnegativity of the conditional 
entropy leads us to a bound for s^™ that matches the message 
size achieved by the constructed code in the limit n — > oo. 

VI. Performance under Erasure Bursts 
OF z Time Steps 

For the third erasure model, we look at erasure patterns 
that contain erasure bursts of a limited number of time steps. 



Consider the first n messages {!,... ,n}, and the union of 
their (overlapping) coding windows T!„. Let £^ be the set of 
erasure patterns in which each erasure burst is z or fewer time 
steps in length, and consecutive bursts are separated by a gap 
of d — z or more unerased time steps, i.e., 

{teE A t+l(^E) \En{t+l,...,t+d-z}\ = 0, 
{t(^E A t+leE) \En{t+l,...,t+z+l}\ < z|. 

The objective is to construct a code that allows all n messages 
{1, . . . ,n} to be decoded by their respective deadlines under 
any erasure pattern E G £f^. Let be the maximum message 
size that can be achieved by such a code, for a given choice 

of (n, c, d, z). 

Using the proof technique of Theorem |2] we can show that 
the constructed intrasession code is asymptotically optimal 
when c? is a multiple of c, or when the maximum erasure 
burst length z is sufficiently short or long: 

Theorem 3. // 

1) d is a multiple of c, or 

2) d is not a multiple of c and z < c — rd.c, or 

3) d is not a multiple of c and z > d — r^^c — Qd,c c, 
then the code constructed in Section \in\ is asymptotically 
optimal in the following sense: the code achieves a message 
size of 



d—z 

E 



which is equal to the asymptotic maximum achievable message 
size lim„_^oc. Sn- 

When the maximum erasure burst length z takes on inter- 
mediate values, intersession coding may become necessary. 
We are currently studying optimal convolutional codes for this 



Appendix A 
Code Properties 

The first property describes when it is possible to decode 
each message: 

Lemma 1 (Achievability). Consider the code constructed in 
Section \ni\for a given choice of (c, d). If message size s 
satisfies the inequality 



i=i 

where y = (yi, . . . ,yd) is as defined in Theorem\l\ then each 
message k G Z+ can be decoded from the data at any I time 
steps in its coding window Wk- 

Note that the maximum message size s that can be supported 
by the code is given by Vj — which corresponds to 

the choice of t = d. 
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(b) (c,d) = (3,8) 



Fig. 3. Partitioning of the set of time steps Tn into tlie d sets T^^"* , . . . , T^i 
in the encoding of messages {1, . . . , 7}, for (a) (c, d) = (3, 9) and 
(b) (c, d) = (3, 8). Each set T,i*' is assigned a unique color. The number i 
at the top of each time step t indicates the set r„ to which t belongs. 



n(d) 



The second property describes a way of partitioning time 
steps into sets with certain specific properties, which are used 
in our specification of the worst-case erasure patterns: 

Lemma 2 (Partition of Coding Windows). Consider the code 
constructed in Section |^/or a given choice of (c, d). Con- 
sider the first n messages {1, . . . , n}, and the union of their 
(overlapping) coding windows Tn- The set of time steps T„ 



can be partitioned into d sets Tn 



(1) 



, ri''^ given by 



{( 



1) 



J q±c 



ifr, 



and — amount of link bandwidth 

<ld,c ■' 



with the following properties: 

1) Over the time steps in the set Tn , each message 
k G {1, . . . , n} is allocated amount of link band- 
width if ri^c < fd 
if ri,c > rd,c- 

2) The allocated link bandwidth in Tn' for each message 
fc G {1, . . . , n} is contained within a single time step 
in its coding window Wk (as opposed to being spread 
over multiple time steps or being outside of the coding 
window). 

3) The total amount of link bandwidth over all time steps 



in In , 1. 1 



J- n 



Wl 



has the following upper bound: 

n 




Fig. [3] shows how the set of time steps T„ is partitioned into 
the d sets Tji'\ . . . , Tn'^\ for two instances of (c, d). 



Appendix B 
Proofs of Theorems 

Proof of Lemma |7} Consider a given message k G Z+ 
and its coding window 

Wk^ {ik-l)c+i:ie{l,...,d}}. 

At each time step t G Wk, message k is allocated either 

1 1 



9d.e + l 



or — ^ amount of link bandwidth; at all other time 

9d,c 



Steps t ^ Wk, message k is allocated zero link bandwidth. 

Let Xi be the amount of link bandwidth at time step t = 
[k — l)c + i that is allocated to message k. Writing t in terms 
of c and Ti c produces 

i = (A: - l)c + i = (/c - 1 + q^^c) c + ■ 

It follows from the code construction that the value of Xi 
depends on 7-^ specifically, two cases are possible: 



Case T. If c 
i G {!,..., d), this 



< Td.c 



then 



Since 



where qi^c e {0, 
message k is allocated 



condition corresponds to the case 
. , qd,c} and r^^c G {1, . . . , rd,c}- Therefore, 



<?d,c + l 



amount of link bandwidth per 
l)rd,c time steps in the coding 



Xj = — . Since 

9d.c 



time step for a total of [qd 
window Wk- 

Case 2: If ^ > T'd,c, then 
i G {1, . . . , d}, this condition corresponds to the case 
where qi^c G {0, . . . , (jd,c - 1} and r^^c e {rd,c + 1, . . . , c}. 
Therefore, message k is allocated ^— amount of link 
bandwidth per time step for a total of qd,c{c^Td.c) time 
steps in the coding window Wk- 

Observe that y is simply a vector containing the elements 
of {xi}f^i sorted in ascending order. Since 



VC/C {!,... 



it follows that over any £ time steps in the coding window Wk, 
the total amount of link bandwidth allocated to message k is 
at least Yfj=iyj- Therefore, as long as the message size s 
does not exceed Vj^ message k can always be decoded 

from the data at any £ time steps in Wk- ■ 
Proof of Lemma ^ The stated partition can be 
constructed by assigning each time step t E Tn to the 



set T, 



, where 



It,. 



qd 



[qd.c + 1) if n^c < rd,c, 

,c if rt.c > Td.c- 



Note that index (7^ ^ c + c G {1, • • ■ , c?} since 
e {0, . . . , gd,c} when r-j^c G {1, ■ • • , ?-d,c}, and 
G {0, ...,gd,c- 1} when r.,,c G {rrf,c + 1, • • ■ , c}. 

To prove the required code properties, we consider two 

separate cases: 
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Case 1: Consider the set T^f^ for a choice of i satisfying 
Ti^c < rd,c- Since each time step t G Tn can be expressed as 



t = {i{ld,c + 1) + qi,c) c + ri^c = tj, where j e Z, 



'0 ' 



9t,c ''t.c 

it follows from the code construction that the set of active 
messages at each time step contains qd,c + 1 messages, and is 
given by 

^t, = 1 j(9rf,c+l)+*,c+l-gd,c,---,i('7d,c+l)+9i,c+lf- 
^ ^ ^ ^ ^ J 



(i) 

The smallest time step in r„ corresponds to the choice of 
j = 0, which produces to — qi,cC + ri,c — i and the set of 
active messages 

Ao = Ui.c + 1 - qd^c, • • • , gix + !}■ 

Note that Atg contains message 1 since q^.c £ {0, . . . , qd,c}, 
which implies that 



Qi.c + 1 - Qd.c < 1 < qi,c + 1- 



At the other extreme, let the largest time step in Tn corre- 
spond to the choice of j = j'; we therefore have 

V <{n-l)c + d< ty+i, (1) 

and the final set of active messages 

= {/(9d,c + l) + '7j,c + l-gdx, ■ • . , i'((7d,c + l)+qix + l}- 
From the first inequality of ([T]i, we obtain 

{fild.c + 1) + qi.c)c + < {n-l + qd,c)c + rd^c 

{fild.c + 1) + Qi.c + 1 - gd,c)c + n^c - Td,, 



n > 



j'iqd,c + 1) + 9j,c + 1 - qd,c + 
= j'(<?d.c + 1) + Qix + 1 - 9d,c, 



(2) 



where the final step follows from the fact that 
1 < Ti^c < fd,c < c, which implies that 



-1 < I — - < '^'^'^ < 



Td,, 



0. 



From the second inequality of ([T]), we obtain 

(n - 1 + qd,c)c + rd,c < ((/+l)(9d,c+l) + '?j,c)c + rj,c - 1 
((/+l)(9d,c+l)+9j,c+l-9d,c)c + rj^c-rd_c-l 



n < 



(/+l)(9d.c+l) + 9^,c + 1 - qd,c + 

/(Q'd.c + 1) + qt.c + 1, 



(3) 



where the final step follows from the fact that 
1 < ri,c < rd^c < c, which implies that 

1 — C— 1 Tir — Tf/r — 1 1 
- 1 = < ^ < < 

c c c 



= -1. 

By combining inequalities (|2]l and (O, we arrive at 

j'iqd,c + 1) + qi,c + 1 - qd,c <n< j'{qd,c + 1) + qi,c + 1, 

which enables us to infer that At ., contains message n. 
For any pair of consecutive time steps tj,tj+i G Tn , where 

tj = {j{qd,c + 1) + qi.c)c + n^c, 
tj+i = {{j + i)('7dx + 1) + qi,c)c + n^c, 

we observe that the smallest message in At is exactly one 
larger than the largest message in At^, i.e.. 



(j + i){qd,c + 1) + qi,c + 1 - qd,c 
= jiqd,c + 1) + qi,c + 1 - qd,c 

= {J{qd.c+l)+q^.c + l) +1- 



qd,, 



Thus, there are no overlapping or omitted messages among the 
sets of active messages corresponding to Tn\ Properties [T] and 
|2] therefore follow. 

The total amount of link bandwidth over all time steps in 
Tn , i.e., \Tn |, can be computed by summing over the link 
bandwidth allocated to the n messages, and adding the unused 
link bandwidth in the smallest time step (which is allocated 
to nonpositive dummy messages) and in the largest time step 
(which is allocated to messages larger than n); this produces 
the required upper bound of Property [3] 

Case 2: Consider the set T„ for a choice of i satisfying 
fi.c > fd^c- Since each time step t £ Tn^ can be expressed as 

t = (j qd,c + qi,c) c+ = tj, where j G Z+, 

9t,c rt,a 

it follows from the code construction that the set of active 
messages at each time step contains qd c messages, and is given 
by 



At, \ j qd,c + qi,c +1 - (qd 



1) 



qt., 




(i) 

The smallest time step in Tl corresponds to the choice of 
j = 0, which produces = qi,c c + c — i and the set of 
active messages 

Ato = {qi,c + 1 - {qdx. - 1), ... , qi,c + 1}. 



Note that At^ contains message 
qi^c e {0, . . . , qd.c - 1}, and therefore 

qi.c + 1 - [qd.c - 1) < 1 < q%.c + 1. 



smce 



At the other extreme, let the largest time step in Tn ' corre- 
spond to the choice of j — j'; we therefore have 



tj' < (n ~ l)c + d < tj'+i, 
and the final set of active messages 



At 



{f qd.c + qi,c + 1 - iqd,c - 1), 



,j qd,, 



(4) 
!}• 



7 



From the first inequality of (01, we obtain 

(/ (ld,c + qi,c)c + Ti^c <{n-l + qd,c)c + rd,, 

if id.c + qi,c + 1 - qd,c)c + n^c - 



n > 



■ f qd.c + qi,c + 1 - qd.c + 



f qd,c + qi.c + 1 - iqd,c - i), 



rd,, 



(5) 



where the final step follows from the fact that 
1 < fd^c < fi^c < c, which implies that 



< 



rd,, 



< 



1 



< 1 



Td,, 



1. 



From the second inequality of (HI, we obtain 

(n - 1 + qd,c)c + Td^c < {W + l)gd.c + qi,c)c + ri,c - 1 

((/+l)9d,c + qt,c + 1 - qd,c)c + n^c - Td^c - 1 



n < 



ij' + l)9d,c + qi,c + 1 - qd^c + 



rd,, 



J qd,, 



qi. 



1, 



(6) 



where the final step follows from the fact that 
1 < fd^c < fi^c < c, which implies that 

1 — 1 r, r — Tf/ r — 1 C — 1 — 1 

= < — — < < 1 



- Td.c - 1 



By combining inequalities (|5]l and (|6]), we arrive at 



= 0. 



i' qd,c + qi.c + 1 - {qd.c -!)<"-< j' qd.c + qi,c + i, 

which enables us to infer that At , contains message n. 

' (i) 

For any pair of consecutive time steps tj,tj+i £ Tn , where 

tj = (j qd,c + qi,c)c + n^c, 

^j+i = iU + 1) (Id.c + qi.c)c + ri,c, 

we observe that the smallest message in At is exactly one 
larger than the largest message in At-, i.e., 



(i + i)'7d,c + qi.c + 1 - (gd,c - 1) 

1) 



(j qd. 



qi.c - 
f qi,c 



1 - iqd.c 
f 1) + 1. 



qd. 



Thus, there are no overlapping or omitted messages among the 
sets of active messages corresponding to Tn \ Properties[T]and 
ID therefore follow. 

The total amount of link bandwidth over all time steps in 

(i) I (i) I 

Tn , I.e., \Tn , can be computed by summing over the link 
bandwidth allocated to the n messages, and adding the unused 
link bandwidth in the smallest time step (which is allocated 
to nonpositive dummy messages) and in the largest time step 
(which is allocated to messages larger than n); this produces 
the required upper bound of Property |3] ■ 



Proof of Theorem Q} Consider the code constructed in 
Section|III]for a given choice of (c, d). According to Lemma[Tl 
if message size s satisfies the inequality 

d~z 

i=i 

then each message k e {1, . . . , n} can be decoded from the 
data at any d — z time steps in its coding window Wk- 
Therefore, the code achieves a message size of X)j=i Vj^ ^y 
allowing all n messages {l,...,n} to be decoded by their 
respective deadlines as long as there are z or fewer erasures 
in each coding window Wk, or equivalently, under any erasure 
pattern E G £™. To demonstrate the asymptotic optimality of 
the code, we will show that this message size matches the 
maximum achievable message size s™ in the limit, i.e.. 



lim s™ 

n— >-oo 



d—z 

i=i 



(7) 



To obtain an upper bound for s™, we consider the cut- 
set bound corresponding to a specific erasure pattern E' from 
£™. Let {1, . . . , d} be partitioned into two sets F^^) and V"^^^ 
where 

F«^{ze{l,...,4:r,,, <rd,,}, 

V^^^ ^{i(.{l,...,d}:r,^,>rd,c]. 

Let V = (vi, . . . ,Vd) be defined as v = (v^^^ | v*^^)), where 
v^^^^ is the vector containing the (g^.c + l)''d,c elements of 
F^^^^ sorted in ascending order, and v*^^^ is the vector contain- 
ing the qd,c{c ~ rd.c) elements of V^^^^ sorted in ascending 
order. Define the erasure pattern E' C r„ as follows; 

d 

E'^ (J T^".), 

j=d-z+l 

where Tn^ is as defined in Lemma |2] The erased time steps 
in E' have been chosen to coincide with the larger blocks 
allocated to each message in the constructed code. To show 
that E' is an admissible erasure pattern, we introduce the 
following lemma: 

Lemma 3. If A C {1, ... ^d}, then 




nw^fe 



^\A\ Vfce{l,...,n}, (8) 



where Tn is as defined in Lemma |2] 



Proof of Lemma |5} Since the code constructed in 
Section |III] allocates a positive amount of link bandwidth to 
each message k E {1, . . . ,n} at every time step in its coding 
window Wk, it follows from Property |2] of Lemma |2] that for 
each i £ {1, . . . , d}, we have 

|rWniyfe| = i Vfce {i,...,n}. 

Equation ([8]) therefore follows from the fact that Ti^\ 
are disjoint sets. 



rp{d) 

J- n 



g 



Applying Lemma[3]with A = {'^jlj^d-z+i produces 

\E'nWk\=z V/c G 

and thus E' is an admissible erasure pattern, i.e., E' e 

Now, consider a code that achieves the maximum message 
size s™. Such a code must allow all n messages {1, . . . , n} to 
be decoded under the specific erasure pattern E' . We therefore 
have the following cut-set bound for s™: 



n4™< 



i=i 



Applying the upper bounds in Property [3] of Lemma |2l and 
writing the resulting expression in terms of yj produces 



d—z 



d—z 



Since a message size of X]j=i Uj known to be achievable 
(by the constructed code), we have the following upper and 
lower bounds for s™: 



1 



d-z 



i=i i=i 
These turn out to be matching bounds in the limit as n - 

d—z ^ d—z d—z 

E ^ 1™ < lim - j;, + 2) = E Vo 



oo: 



We therefore have O as required. ■ 
Proof of Theorem |2} Consider the code constructed in 
Section Unlfor a given choice of (c, d). According to Lemma[Tl 
if message size s satisfies the inequality 

d—z 

s < E^'j' 

then each message k G {1, . . . , n} can be decoded from the 
data at any d — z time steps in its coding window Wk ■ 
Therefore, the code achieves a message size of X]j=i 
by allowing all n messages to be decoded by 

their respective deadlines as long as there are z or fewer 
erasures in each coding window W^, which is indeed the case 
when there are z or fewer erasures in each sliding window 
Lt, or equivalently, under any erasure pattern E G £^ . To 
demonstrate the asymptotic optimality of the code, we will 
show that this message size matches the maximum achievable 
message size s^™ in the limit, i.e.. 



d—z 



lim 4" = E2^^- 



(9) 



We consider two cases separately, depending on whether d is 
a multiple of c: 



Case 1: Suppose that d is a multiple of c. In this case, the 
message size achieved by the constructed code simplifies to 



d—z 

E 



d — z 



d- 



Vj = 



Qd.c 



1 



To obtain an upper bound for sf", we consider the cut- 
set bound corresponding to a specific periodic erasure pattern 
E' C T„ given by 



E' 



{jd + iGT„:jGZ+,iG{l,...,z}}. 



Since E' comprises alternating intervals of z erased time steps 
and d — z unerased time steps, we have exactly z erasures in 
each sliding window Lt; therefore, E' is an admissible erasure 
pattern, i.e., E' G 

Now, consider a code that achieves the maximum message 
size s^™. Such a code must allow all n messages {1, . . . , n} to 
be decoded under the specific erasure pattern E' . We therefore 
have the following cut-set bound for s^: 

n-l)c + d 



n4:<\Tn\E' 
where 



{d — z) + max(r' — z, 0), 



r' = (n — l)c + d — 
Further simplification produces 



(n -l)c + d 



1 {n - l)c+2d 



(d-z) 



d — z 



d. 



2d 



Since a message size of is known to be achievable (by 
the constructed code), we have the following upper and lower 
bounds for s^": 



d- 



-c < 4" < 



d- 



2d-: 



d - - d \ n 
These turn out to be matching bounds in the limit as n — >^ cxd: 

c\ d — z 



z I 2d 
c+ 



'^—T^c < lim < lim 

We therefore have ^ as requked. 

Case 2: Suppose that d is not a multiple of c. Consider a 
specific base erasure pattern E' C r„ given by 

i?'= U T^), 

j=d-z+l 

where Tn^ is as defined in Lemma|2] and v = (wi, . . . , Vd) is 
as defined in the proof of Theorem [T] The erased time steps 
in E' have been chosen to coincide with the larger blocks 
allocated to each message in the constructed code. From E', 
we derive the erasure patterns E'l, . . . , E'^ given by 

d 

El^E'nWk= U (Ti''^'> n Wk'^ 

j=d-z+l 

Applying Lemma [3] with A = produces 



\El\ 



\E'nWk\=z V/c G 
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which implies that 

\E'k^Lt\<z Vi e {l,...,(n-l)c+l} 

for each fc G {1, . . . , n}. Thus, E'^. is an admissible erasure 
pattern, i.e., £ f for each fc e {1, . . . , n}. 

To obtain an upper bound for s^™, we introduce the follow- 
ing lemma: 

Lemma 4. Suppose that a code achieves a message size of 
s under a given set of erasure patterns £ for a given choice 
of (n, c, d). Let Xt be a random variable representing the 
coded data transmitted at time step t G T„, let be a 
random variable representing message fcG{l,...,n}, and 
define X[A\ = {Xt)t£A- If E C Tn is such that E n Wk is 
an admissible erasure pattern, i.e., [E fl Wk) G £, for each 
k G {1, . . . , n\, then for each k G {1, . . . , n}. 



H(^X[Wk\E] I Aff,xf < \Tk\E\ -ks 



(10) 



Proof of Lemma |5} We will prove by induction that 
inequality ( fTOl i holds for any k G {l,...,n}. 

(Base case) Consider the case of fc = 1. From the definition 
of mutual information, we have 

l{X[Wi\E] ; Ml) = H{X[Wi\E]) - H{X[Wi\E] \ Mi) 
= H{Mi) - H{Mi I X[Wi\E]). 

Rearranging terms produces 

H{X[Wi\E] I Ml) = H{X[Wi\E]) - H{Mi) 

+ H{Mi\X[Wi\E]). (11) 

Since Ti — Wi and H{Xt) < 1 for any t because of the unit 
link bandwidth, we have 

H{X[Wi\E]) ^ H{X[Ti\E]) < \Ti\E\. (12) 

Furthermore, since E Wi is an admissible erasure pattern, 
message 1 must be decodable from the coded data at time 
steps Ti\{E n Wi) = Wi\{E n Wi) = Wi\E, and so 

H{Mi\X[Wi\E]) ^0. (13) 

Substituting ([T2]i, O, and H{Mi) = s into (dB yields 

H{X[Wi\E] I Ml) < \Ti\E\ - s, 

as required. 

(Inductive step) Suppose that 

H(^X[Wk\E] I M^xf < \Tk\E\ -ks (14) 

for some k G — 1}. From the definition of condi- 

tional mutual information, we have 



l(^X[Wk+i\E] ; Mk+i 
= H(x[Wk+i\E] M^X^ 



H(x[Wk+i\E] 



M^+\X^' 



-H(^Mk+i\M^,X[{l, 
Rearranging terms produces 



..,kc}UiWk+i\E)]y 



H(x[Wk+i\E] 



M^+^,X^' 
H(x[Wk+i\E] M^,X^- 
H(Mk+i M^X^' 



H(Mk+i 



MlX[{l,...M]^{Wk+i\E)] 



(15) 



Since messages are independent and message fc + 1 is created 
at time step fee + 1, we have 

H(Mk+i I M'l, Xf) = H{Mk+i) = s. (16) 

Furthermore, since E n Wk+i is an admissible erasure pattern, 
message fc + 1 must be decodable from the coded data at 
time steps Tk+i\{E n Wk+i) = {Tk+i\Wk+i) U {Wk+i\E) 
= {l,...,kc:\{j{Wk+i\E), and so 



H(Mk+i I Aff, . . . , fee} U {Wk+i\E)]^ 

Substituting ([T6]l and into ([T5]l yields 

H(x[Wk+i\E]\M'l+\x'l^ 
^H(x[Wk+i\E]\Mtx\ 

'(x[{Wk\E) U {Wk+i\E)] I M'l.X'l^ 



0. (17) 



(a) , 



(b) 



< H[X[{Wk\E)\j{Wk+i\E)] 

^<H(x[Wk\E] Af{=,x}''~^^") 
+ H(x[{Wk+i\E)\{Wk\E)] 



M^,X{ 



(fe-l)c 



(d) 



<\Tk\E\-ks+ {Wk+i\E)\{Wk\E) 

^^\Tk+i\E\-{k + l)s, 
as required, where 

(a) follows from the addition of random variables X[Wfe\i?] 
in the entropy term; 

(b) follows from the removal of conditioned random variables 
■^(k-i)c+i entropy term; 

(c) follows from the chain rule for joint entropy, and the 
removal of conditioned random variables Af^', 



and X\ 



{k-l)c 



in the second entropy term; 



(d) follows from the inductive hypothesis ( fT4l ). and the fact 
that H{Xt) < 1 for any t because of the unit link 
bandwidth; 

(e) follows from the fact that 

\Tk\E\ + \{Wk+i\E)\{Wk\E)\ 
= \Tk\E\ + \{Wk+i\Wk)\E\ 
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= \Tk\E\ + \{Tk+i\Tk)\E\ 
= \Tk+i\E\. 

■ 

Applying Lemma|4]with £ — and E — E' to an optimal 
code that achieves a message size of s^™ produces 



H[X[Wk\E'] M^\X\ 



,(fc-i)( 



< \Tk\E'\~ksl 



for any fc e {1, . . . , n}. Since the conditional entropy term is 
nonnegative, it follows that for the choice of fc = n, we have 

\T^\E'\-ns'^ > ^ 4™ < -\T^\E'\ - 1 V |T^)|. 
II n' ' n ^ — ' ' ' 

Applying the upper bounds in Property [3] of Lemma |2] and 
writing the resulting expression in terms of yj produces 

n ^ — ^ ' ' n ^ — ^ 



Since a message size of X]j=i % known to be achievable 
(by the constructed code), we have the following upper and 
lower bounds for s!": 



d—z 



d—z 



^=1 j=i 

These turn out to be matching bounds in the limit as n — >^ cxd: 



d—z 



i=i i=i 
We therefore have (|9]l as required. ■ 
Proof of Theorem \3\ Observe that under each erasure 
pattern E G the coding window Wk for each message 
k S {1, . . . , n} contains at most z erasures: if Wk intersects 
with zero erasure bursts, then it contains zero erasures; if Wk 
intersects with exactly one erasure burst, then it contains at 
most z erasures, i.e., the maximum length of a burst; if Wk 
intersects with two or more erasure bursts, then it contains a 
gap of at least d — z unerased time steps between consecutive 
bursts, and therefore contains at most z erasures. 

Consider the code constructed in Section |lll] for a given 
choice of (c, d). According to Lemma [T] if message size s 
satisfies the inequality 

d—z 

s < E^j' 

i=i 

then each message k £ {1, . . . , n} can be decoded from the 
data at any d — z time steps in its coding window Wk ■ 
Therefore, the code achieves a message size of X]^=i 
allowing all n messages to be decoded by their 

respective deadlines as long as there are z or fewer erasures 
in each coding window Wk, which is indeed the case under 
any erasure pattern E G To demonstrate the asymptotic 
optimality of the code, we will show that this message size 



matches the maximum achievable message size s^j in the limit, 
i.e., 



lim 



E^j' 

i=i 



(18) 



for the following three cases: 

Case 1: Suppose that d is a multiple of c. In this case, the 
message size achieved by the constructed code simplifies to 



d~z 
= 1 



qd,c 



1 



To obtain an upper bound for s^, we consider the cut-set bound 
corresponding to a specific periodic erasure pattern E' C r„ 
given by 

E' ^{jd + ieTn-.j ez+,ie{i,..., z}}. 

Since E' comprises alternating intervals of z erased time steps 
and d—z unerased time steps, it is an admissible erasure 
pattern, i.e., E' e £f^. 

The rest of the proof leading to the obtainment of ( fTSl ) is 
the same as that of Case 1 in the proof of Theorem |2] with 
s^™ replaced by s^. 

Case 2: Suppose that d is not a multiple of c, and 
z < c — rd,c- In this case, the message size achieved by the 
constructed code simplifies to 



d-z 

E^j 



d 

c - E = c 

j=d-z+l 



Consider a specific base erasure pattern E' C r„ given by 



E' ^ U T, 

j=d-z+l 



where T^'^ is as defined in Lemma|2] and v = (wi, . . . , Vd) is 
as defined in the proof of Theorem [1] The erased time steps 
in E' have been chosen to coincide with the larger blocks 
allocated to each message in the constructed code. In this case, 
E' simplifies to 



-n((9d,c-l)c+ri,c) 



E' = U TA 

Ti.c— c — 2:+l 

= {((i + i)%,c-i)c + ri,c eT„ 



which follows from the definition of Tn^ and the fact that 
?'i,c > frf.c when ri^c G {c — z + 1, . . . , c}. Observe that E' 
comprises alternating intervals of z erased time steps and 
qd.c c — z unerased time steps, with each interval of erased 
time steps corresponding to a specific choice of j G Z([ . Since 
each erased time step t € E' can be expressed as 

t= {{j + '^)qd,c-l) c+r,^cj 

Qt.c Tt,o 
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it follows from Section |III] that the set of active messages At 
at time step t is given by 



At 



{ (j + l)g<i,c -{qd,c - 1), . . . , (j + l)qd^c }. 



9t,c + l <?t,c + l 

Therefore, the set of active messages At is the same at 
every time step t in a given interval of z erased time steps 
(corresponding to a specific j). 

From E', we derive the erasure patterns E[, . . . , given 

by 

d 

j=d-z+l 

Applying Lemma[3]with A = produces 

\El\ = \E'nWk\= z y k e {l,...,n}. 

Let t' G E'^ be one of the z erased time steps in Wk under 
erasure pattern E'^.. As previously established, t' belongs to an 
interval of z erased time steps in E' that have the same set 
of active messages Af (which contains message k). It follows 
that this interval of z erased time steps is also in E'^, and must 
therefore constitute E'^ itself. Thus, E'^. is an admissible era- 
sure pattern, i.e., E'^. € for each k S {1, . . . , n}, because 
it comprises a single erasure burst of z time steps. 

Applying Lemma |4] with £ = £^ and E = E' to an optimal 
code that achieves a message size of s^^ produces 



H(x[Wk\E'] 



< \Tk\E'\-ksl 



for any fee {!,..., n}. Since the conditional entropy term is 
nonnegative, it follows that for the choice of A: = n, we have 



\Tn\E'\ 



nsl>0 



< 



1 



Tn\E'\ 



d — z 

E 



The rest of the proof leading to the obtainment of ( fTSl ) is the 
same as that of Case 2 in the proof of Theorem |2] with s^™ 
replaced by s^^. 

Case 3: Suppose that d is not a multiple of c, and 
z > d — rd^c = <ld,c c. In this case, the message size achieved 
by the constructed code simplifies to 

d—z J 

Ed — z 

Consider a specific base erasure pattern E' C T„ given by 

j=d~z+l 

where Tn^ is as defined in Lemma |2] and v = (wi, . . . , Vd) is 
as defined in the proof of Theorem [T] The erased time steps 
in E' have been chosen to coincide with the larger blocks 
allocated to each message in the constructed code. In this case, 
E' simplifies to 



= Tn^^{j{qd,c + l))c + n^c e Tn : 

ieZ+ r,,,e{i,...,d-z}}, 

which follows from the definition of T,!'^ and the fact that 
ri.c < 'i'd,c when c G {1, . . . ,d — z}. Observe that E' com- 
prises alternating intervals of d ~ z unerased time steps and 
{qd.c + I) c — {d — z) = c — Td^c + z erased time steps, with 
each interval of unerased time steps corresponding to a specific 
choice of j G Zq. Since each unerased time step t G Tn\E' 
can be expressed as 

t = {j{qd,c + 1)) c+ , 

qt,c rt,a 

it follows from Section Hill that the set of active messages At 
at time step t is given by 



At = 



{ j{qd,c + 1) +1 - qd.c. ■ ■ . , j(gd,c + 1) 



Therefore, the set of active messages At is the same at every 
time step t in a given interval of d — z unerased time steps 
(corresponding to a specific j). 

From £", we derive the erasure patterns E[,...,E'^ given 

by 

d 

Ei^E'nWk^ U (t^'^ n Wk) ■ 

j=d-z+l 

Applying Lemma [3] with A = {vj}j^d-z+i produces 

\El\ = \E' nWk\= z V/c G {!,..., n}. 

Let t' G Wk\E'i^ be one of the d — z unerased time steps in 
Wk under erasure pattern E'/^. As previously established, t' 
belongs to an interval of d — z unerased time steps in Tn\E' 
that have the same set of active messages At/ (which contains 
message k). It follows that this interval of c? — z unerased time 
steps is also in Wk\E'^., and must therefore constitute Wk\E'^. 
itself. Thus, E'^ is an admissible erasure pattern, i.e., E'^. G £^, 
for each k G {!,..., n], because it comprises either a single 
erasure burst of z time steps, or two erasure bursts with a 
combined length of z time steps separated by a gap of d — z 
unerased time steps. 

Applying Lemma |4] with £ = £f^ and E = E' to an optimal 
code that achieves a message size of produces 



H[X[Wk\E'] Mf,X} 



< \Tk\E'\-ksl 



for any fc G {1, . . . , n}. Since the conditional entropy term is 
nonnegative, it follows that for the choice of A: = n, we have 



d—z 



\T^\E'\-n.sl > ^ 4 < -|T„\£;'| = -J2K- 



The rest of the proof leading to the obtainment of (fTSI l is the 
same as that of Case 2 in the proof of Theorem |2] with sf™ 
replaced by s^ . ■ 



12 



References 

[1] D. Leong and T. Ho, "Erasure coding for real-time streaming," in Proc. 
IEEE Int. Symp. Inf. Theory (ISIT), Cambridge, Massachusetts, USA, Jul. 
2012. 

[2] E. Martinian and C.-E. W. Sundberg, "Low delay burst erasure correction 
codes," in Proc. IEEE Int. Conf. Commm. (ICC), May 2002, pp. 1736- 
1740. 

[3] E. Martinian and M. Trott, "Delay-optimal burst erasure code construc- 
tion," in Pwc. IEEE Int. Symp. Inf. Theory (ISIT), Jun. 2007, pp. 1006- 
1010. 

[4] L. J. Schulman, "Coding for interactive conununication," IEEE Trans. Inf 

Theory, vol. 42, no. 6, pp. 1745-1756, Nov. 1996. 
[5] A. Sahai, "Anytime information theory," Ph.D. dissertation, Massachusetts 

Institute of Technology, 2001. 
[6] R. T. Sukhavasi, "Distributed control and computing: Optimal estimation, 

error correcting codes, and interactive protocols," Ph.D. dissertation, 

California Institute of Technology, 2012. 
[7] O. F. Tekin, S. Vyetrenko, T. Ho, and H. Yao, "Erasure correction 

for nested receivers," in Proc. Armu. Allerton Conf. Commun., Control, 

Comput. (Allerton), Sep. 2011, pp. 1454-1461. 



